DOCUMENT RESUME 



ED 411 156 



SE 060 553 



TITLE 



INSTITUTION 

ISBN 

PUB DATE 
NOTE 
PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Achievements of Primary 4 and Primary 5 Pupils in 
Mathematics and Science. Third International Mathematics and 
Science Study. 

Scottish Office Education and Industry Dept . # Edinburgh. 
ISBN-0-7480-5838-9 
1997-00-00 
2 9p . 

Reports - Descriptive (141) 

MF01/PC02 Plus Postage. 

* Academic Achievement; Academic Standards; Foreign 
Countries; Intermediate Grades; International Studies; 
Mathematics Curriculum; ^Mathematics Tests; Science 
Instruction; *Science Tests; *Student Surveys 
Scotland; Third International Mathematics and Science Study 



ABSTRACT 



This document reports on the performance of Scottish primary 
students in mathematics and science. Students scored below the international 
average in mathematics and are ranked in the lower half of the range of 
countries involved in the study. In science, students scored above the 
international average and are ranked in the middle. Sections addressing 
performance in mathematics, examples of mathematics test items, Scottish 
features and international comparisons in mathematics, performance in 
science, examples of science test items, and Scottish features and 
international comparisons in science are included. (DDR) 



*********** ******* ************* *********** 



*********************************** 



Reproductions supplied by EDRS are the best that can be made 
from the oricrinal document. 



ED 411 156 




THE SCOTTISH OFFICE 

Education and Industry Department 



Achievements of 
Primary 4 and 
Primary 5 Pupils 
in Mathematics 
and Science 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL 
HASJ3EEN GRANTED BY 



TO 

k 



THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



1 1 q HPPARTMENT of education 

auCATIONAL RESOURCES INFORMATION 
r CENTER (ERIC) 

[ This document has been teP r °d“F?° f s 
Seceived from the person or organization 
originating it. 

I Minor changes have been made to 
improve reproduction quality. 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 




BEST COPY AVAILABLE 

Third International Mathematics 
and Science Study (TIMSS) 



2 



THE SCOTTISH OFFICE 



Education and Industry Department 



Achievements of 
Primary 4 and 
Primary 5 Pupils 
in Mathematics 
and Science 



Third International Mathematics 
and Science Study (TIMSS) 



© Copyright The Scottish Office Education and Industry Department, 1997 
ISBN: 0 7480 5838 9 




Contents 



Page 



1 Introduction 1 

Sampling 1 

Administration 1 

Assessment instruments 2 

MATHEMATICS 

2.1 Performance in mathematics 3 

Overall mean scores 3 

Differences between the lower and upper grades 4 

Benchmarks of performance 4 

Curriculum matching exercise 4 

Performance of 9 year olds 4 

Performance in different aspects of mathematics 5 

Gender and performance in mathematics 5 

2.2 Examples of mathematics test items 7 

2.3 Scottish features and international comparisons 13 



in mathematics 

SCIENCE 



3.1 Performance in science 16 

Overall mean scores 16 

Differences between the lower and upper grades 1 7 

Benchmarks of performance 17 

Curriculum matching exercise 1 8 

Performance of 9 year olds 1 8 

Performance in different aspects of science 18 

Gender and performance in science 1 8 

3.2 Examples of science test items 19 

3.3 Scottish features and international comparisons 22 

in science 

4 Conclusions 24 




5 



THIRD INTERNATIONAL MATHEMATICS AND SCIENCE STUDY 



7 Introduction 



The Third International Mathematics and Science 
Study (TIMSS), carried out in 1 995, was the latest in 
a series of international studies carried out by the 
International Association for the Evaluation of 
Educational Achievement (IEA). In the past the IEA 
carried out separate international studies in the two 
subjects - in 1964 and 1980-82 in mathematics and 
1970-71 and 1983-84 in science. These studies were 
of the attainment of 10, 13 and 14 year olds and on 
pupils in their final year at school. The International 
Assessment of Educational Progress (IAEP) also 
carried out a study of attainment in mathematics and 
science of 9 and 1 3 year olds in 1991. 

Scotland is one of over 40 countries which 
participated in TIMSS, including most of the EU 
member states, other major English speaking 
countries, various Pacific rim countries and a number 
of Eastern European countries. 

The main aims of TIMSS were to collect 
information about: 

□ pupils’ knowledge and understanding of 
mathematics and science; 

□ mathematics and science curricula in the 
participating countries; 

□ teaching and learning practices in 
mathematics and science. 

In order to achieve these aims, pupils were 
administered tests in mathematics and science, an 
analysis of curriculum guides and textbooks was 
carried out and questionnaires were completed by 
schools, teachers and pupils. 

This report summarises the TIMSS results for 
primary 4 and primary 5 pupils in Scotland. A full 
listing of the international reports can be found on the 
World Wide Web: http://wwwcsteep.bc.edu/timss. 

Sampling 

Sampling was a two stage process involving first the 
random selection of schools, then the random selection 

O 




of a class at each of the two grades containing the 
most 9 year olds. In Scotland these grades are primary 
4 and primary 5. Details of the sampling procedure 
are provided in the TIMSS Technical Report Volume 
1: Design and Development. 

The overall sample for each country for TIMSS 
was 150 but, due to the small size of many of our 
primary schools , the Scottish target was 1 84. For the 
lower grade the response rate was 81% and for the 
upper grade it was 83%, when replacement schools 
were included. The target samples of pupils were 
3,476 at the lower grade and 3,596 at the upper grade 
and response rates of 90% and 92% respectively were 
obtained. 

Scottish pupils were the third youngest of all the 
countries; pupils were slightly younger in Iceland 
and Greece. However, along with England, New 
Zealand, Australia and Kuwait, they had the most 
years of formal schooling. 

Administration 

The tests and questionnaires were sent out to schools 
in April 1995. The tests were administered by the 
schools and were timed to allow the majority of 
pupils to complete them. The pupils then completed 
their questionnaires. School questionnaires were 
expected to be completed by the head teacher or his/ 
her deputy and the teacher questionnaires were 
completed by the teachers of the classes that were 
tested. 

The pupil questionnaire asked about home 
background, attitudes towards mathematics and 
science and perceptions of lessons in these subjects. 
The teacher questionnaire asked about education, 
training and experience, how teachers divided their 
time between teaching and teaching related activities, 
teaching approaches, resources available and views 
on teaching and learning in mathematics and science. 
The questionnaire for schools asked for background 
information about the school, organisational features 
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related to the teaching of mathematics and science, 
the availability of resources and the time devoted to 
the teaching of these subjects. 

Assessment instruments 

The assessments were in the form of written tests. 
Each pupil sat a test in one booklet, out of eight 
booklets in total, which contained mathematics and 
science tasks. All tests contained a common core of 
tasks, with the other tasks appearing in different 
versions of the test. Tests were allocated to pupils in 
rotation, so only a few pupils in each school took the 
same test. The total test battery consisted of 102 
mathematics tasks and 97 science tasks. The tests 
were administered in two sessions taking 
approximately 60 minutes in total. 

The same tasks were used in all countries in order 
that comparisons could be made of their relative 
performance. However, in addition, a curriculum 
matching exercise was carried out which involved 
each country in defining what tasks were covered in 
its curriculum. A further analysis was then done for 
all countries based only on these tasks to provide 
performance measures relevant to that country. 



The tasks included multiple choice questions, 
short answer questions and extended response 
questions. About a quarter of the tasks were in a free 
response format. The reporting categories used in 
this report are: 

Mathematics 

□ whole numbers 

□ fractions and proportionality 

□ measurement, estimation and number sense 

□ data representation, analysis and 
probability 

□ geometry 

□ patterns, relations and functions 
Science 

□ earth science 

□ life science 

□ physical science 

□ environmental issues and the nature of 
science 
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2 b I Performance in mathematics 



This report compares the mathematics performance 
of Scottish pupils with that of pupils in the other 
countries participating in TIMSS. The comparisons 
are made in terms of: 

□ overall mean scaled scores 

□ mean scores in the six reporting categories 

□ mean scores on the tasks identified in the 
curriculum matching exercise 

□ performance on selected individual tasks 

The grades selected for this phase of testing in 
TIMSS were the two grades containing the greatest 
number of 9 year olds. In Scotland these grades were 
primary 4 and primary 5. Separate mean scores were 
calculated for pupils at each grade and median scores 
for pupils at both grades who were 9 years old at the 
time of testing for countries in which the two grades 
contained at least 75% of nine year olds. Where 
appropriate, comparisons are made with the 1991 
IAEP results. Unless otherwise stated, 24 countries 
took part in the lower grade tests and 26 countries in 
the upper grade tests. In comparing the results from 
the different countries it should be borne in mind that 
Scottish pupils were amongst the youngest pupils in 
the study, although they had more years of formal 
schooling than pupils in most other countries. 

Overall mean scores 

The international scores for the lower and upper 
grades in all countries are presented in Table 1 . These 
are scaled scores calculated using plausible values 
procedures which are explained in the second TIMSS 
Technical Report. The overall mean score for the 
lower grade was 470, compared to Scotland’s 458, 
and for the upper grade it was 529, compared to 
Scotland’s 520. Overall Scotland’s ranking was 15th 
out of 24 countries at the lower grade and 1 6th out of 
26 countries at the upper grade, including countries 
which did not meet TIMSS sampling criteria. 
However, comparisons with countries which did not 



Table 1 : Mean scaled scores in mathematics for 
each grade. 



Country 


Mean Score 


Mean Score 




Lower Grade 


Upper Grade 


Korea 


561 


611 


Singapore 


552 


625 


Japan 


538 


597 


Hong Kong 


524 


587 


Czech Republ 


ic 497 


567 


Netherlands 


493 


577 


Austria 


487 


559 


Australia 


483 


546 


USA 


480 


545 


Ireland 


476 


550 


Canada 


469 


532 


Latvia 


463 


525 


Scotland 


458 


520 


England 


456 


513 


New Zealand 


440 


499 


Cyprus 


430 


502 


Greece 


428 


492 


Portugal 


425 


475 


Norway 


421 


502 


Iceland 


410 


474 


Iran 


378 


429 


Countries not 


meeting TIMSS sampling criteria 


Slovenia 


488 


552 


Hungary 


476 


548 


Israel 


- 


531 


Thailand 


444 


490 


Kuwait 


__ 


400 



meet the sampling criteria should be treated with 
caution. 

Most of the Pacific rim countries did very well in 
mathematics, with Korea, Singapore, Japan and Hong 
Kong leading in the performance table. Thailand was 
the exception and was rated 17th and 22nd at the 
lower and upper grades respectively. 

Of the European countries, The Netherlands, 
Austria and Ireland did well. England was not 
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significantly different from Scotland, and Greece, 
Portugal, Norway, Cyprus and Iceland performed 
more poorly. 

Of the four Eastern European countries, the Czech 
Republic, Slovenia and Hungary performed well, 
and Latvia performed very similarly to Scotland. 

Other English speaking countries varied in their 
performance. Australia and the USA performed quite 
well, Canada performed about the same as Scotland 
and New Zealand performed more poorly than 
Scotland. 

Table 2 shows Scotland’s position relative to 
other countries in statistical terms. Twelve countries 
performed significantly better than Scotland and four 
countries performed the same as Scotland at both 
grades. The remaining seven countries at the lower 
grade and nine countries at the upper grade performed 
significantly more poorly than Scotland. 

We can make a few comparisons with IAEP 
carried out in 1991. Of the countries which participated 
in TIMSS and in the IAEP, all were rated similarly in 
both studies in relation to Scotland except Slovenia 
and the USA, which performed more poorly than 
Scotland in IAEP and better than Scotland in TIMSS. 

Differences between the lower and 
upper grades 

The same tests were taken by each grade and the 
differences in scores between the lower and upper 
grade gives an indication of the gains made in this 
year. In Scotland the gain in score was 62, compared 
to the international average gain of 59. Scotland's 
gain was the 16th greatest of the 24 countries. The 
differences between grades varied from 81 (in 
Norway) to 46 (in Thailand). 

If we look at the mean differences in scores in the 
different aspects of mathematics in Scotland, we see 
that the greatest gain was in data representation, 
analysis and probability and the least gain was in 
geometry. 

If we regard the average gain of 59 score points 
between the grades as representing a year’s progress, 
then four countries at both the lower and upper grades 



can be regarded as being more than a year ahead of 
Scotland in mathematics. They are Hong Kong, 
Singapore, Japan and Korea. 

Benchmarks of performance 

If we regard the mean scores over all countries for the 
top 10%, the top 25% and the top 50% of pupils as 
benchmarks of performance, we can compare them 
with the percentage of pupils in each country reaching 
these benchmarks. At the lower grade, 5% of Scottish 
pupils attained the 10% benchmark, 16% attained the 
25% benchmark and 4 1 % attained the 50% benchmark 
and Scotland was ranked 1 6th for all three benchmarks. 
At the upper grade the equivalent percentages were 
6%, 18% and 43% for the three benchmarks and 
Scotland was ranked 15th, 16th and 16th respectively 
on each of the benchmarks. 

Curriculum matching exercise 

In the curriculum matching exercise, the TIMSS 
tasks were scanned by each country to identify those 
which were within their curriculum. The performance 
of pupils was then assessed on the basis of the tasks 
judged to be within the curriculum of each country. 
The tasks identified as being within Scotland’s 
curriculum had total unsealed scores of 46 score 
points at the lower grade and 92 score points at the 
upper grade, compared to the total of 1 1 3 score points 
for all of the tasks at both grades. Scotland’s scores 
were 54% at the lower grade and 60% at the upper 
grade on the selected tasks and her rankings were 
1 3th at both grades (out of 22 countries at the lower 
grade and 24 countries at the upper grade). This was 
only a slight improvement on Scotland’s ranking on 
all the test items which suggests the TIMSS tests 
were well matched to our curriculum. 

Performance of 9 year olds 

In 22 of the TIMSS countries, the percentage of 9 year 
olds in the two grades tested exceeded 75%. For these 
countries the median scores have been taken as the 
best indicator of performance. Scotland’s ranking in 
terms of her median score was 8th, a major 
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Table 2: Scotland's position relative to other countries 





Lower Grade 


Upper Grade 


Countries scoring significantly 
higher than Scotland 


Korea, Singapore, Japan, 

Hong Kong, Czech Republic, 
Netherlands, Slovenia, Austria, 
Australia, USA, Hungary, 
Ireland 


Singapore, Korea, Japan, 
Hong Kong, Netherlands, 
Czech Republic, Austria, 
Slovenia, Ireland, Hungary, 
Australia, USA 


No significant difference 
from Scotland 


Canada, Latvia, England, 
Thailand 


Canada, Israel, Latvia, 
England 


Countries scoring significantly 
lower than Scotland 


New Zealand, Cyprus, 
Greece, Portugal, Norway, 
Iran, Iceland 


Cyprus, Norway, New Zealand, 
Greece, Thailand, Portugal, 
Iceland, Iran, Kuwait 



improvement on her overall rankings due to the 
younger age of the Scottish pupils compared to the 
pupils in nearly every other country. 

Performance in different aspects of 
mathematics 

There were different numbers of tasks in the different 



aspects of mathematics: 

Whole numbers 25 tasks 

Fractions and proportionality 21 tasks 

Measurement, estimation 

and number sense 20 tasks 

Data representation, 

analysis and probability 12 tasks 

Geometry 14 tasks 

Patterns, relations and functions 10 tasks 







The performance of Scottish pupils varied in the 
different aspects of mathematics covered by the tests. 
The differences are exemplified in Table 3 by showing 
Scotland’s ranking in the different aspects. 
Performance was relatively poor on whole numbers, 
relatively good on data representation, analysis and 
probability and very good on geometry. 

Gender and performance in 
mathematics 

At the lower grade, boys did better than girls in 19 
countries out of 24, but the differences were only 
significant in six of them. At the upper grade, boys 
did better than girls in 16 countries out of 25, but the 
differences were only significant in three of them. In 
Scotland, boys scored higher than girls at the lower 
grade, but the difference was not significant. At the 
upper grade, boys and girls scored the same. 



io 
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Table 3: Scotland’s rankings on aspects of mathematics 



Aspect 


Lower grade ranking 


Upper grade ranking 


Whole numbers 


16 


18 


Fractions and proportionality 


14 


16 


Measurement, estimation and 
number sense 


14 


16 


Data representation, analysis 
and probability 


13 


1 1 


Geometry 


2 


4 


Patterns, relations and functions 


15 


16 



O 

ERIC 
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2.2 Examples of mathematics test items 

Whole numbers 



Example 1 : Choose the largest number 

Scotland's performance 



Which of these is the largest number? 




Percentage 


International 






correct 


ranking 


A. 2735 


Lower grade 


68 


18 


B. 2537 


Upper grade 


85 


18 


C. 2573 








D. 2753 









Example 2: Complete number sentence 



Scotland's performance 



Here is a number sequence. 



2000 + Q + 30 + 9 = 2739 



What number goes where the 
sentence true? 



□ 



is to make this 



Percentage International 
correct ranking 

Lower grade 2 1 22 

Upper grade 47 23 



Answer: 



Fractions and proportionality 



Example 3: Longest box on shelf 

Julie put a box on a shelf that is 96.4 centimeters long. 
The box is 33.2 centimeters long. What is the longest 
box she could put on the rest of the shelf? Show all 
your work. 

Answer: 



Scotland’s performance 

Percentage International 
correct ranking 

Lower grade 8 1 2 

Upper grade 27 12 



0 

ERIC 
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Example 4: Fraction of figure shaded 



Scotland’s performance 



Part of the figure is shaded. 



Percentage International 
correct ranking 

Lower grade 40 1 1 

Upper grade 66 13 



What fraction of the figure is shaded? 



A. 



5 

4 



B. 



4 

5 



C. 



6 

9 



D. 



5 

9 



Measurement, estimation and number sense 



Example 5: Best estimate of clothespin mass 



Scotland's performance 



The weight (mass) of a clothespin is 9.2 g. Which of 
these is the best estimate of the total weight (mass) of 
1000 clothespins? 



A. 900 g 

B. 9,000 g 

C. 90,000 g 

D. 900,000 g 



Percentage International 
correct ranking 

Lower grade 40 1 3 

Upper grade 50 15 
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Example 6: Length of rectangle 

Scotland's performance 



A thin wire 20 centimeters long is formed into a rectangle. 




Percentage 


International 


If the width of this rectangle is 4 centimeters, what is its 
length? 




correct 


ranking 


Lower grade 


26 


4 


A. 5 centimeters 

B. 6 centimeters 

C. 12 centimeters 

D. 16 centimeters 


Upper grade 


24 


8 



Data representation, analysis and probability 



Example 7: Bar graph: cartons sold on Monday 

The graph shows the number of cartons of milk sold each 
day of a week at a school. 




Day 



Scotland’s performance 

Percentage International 
correct ranking 

Lower grade 63 1 0 

Upper grade 83 10 



How many cartons of milk did the school sell on Monday? 

Answer: 

How many cartons of milk did the school sell that week? 
Show your work. 

Answer: 



O 

ERLC 
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Example 8: Pictograph of trees 



Scotland's performance 




Percentage International 
correct ranking 

Lower grade 35 8 

Upper grade 56 8 



How many trees does each 



represent? 



Answer: 



Geometry 



Example 9: Cut out shape 

Craig folded a piece of paper in half and cut out a shape. 




Scotland's performance 

Percentage International 
correct ranking 

Lower grade 60 6 

Upper grade 76 5 



Draw a picture to show what the cut-out shape will look 
like when it is opened up and flattened out. 



0 

ERIC 
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Example 1 0: Edges of cube 



Scotland's performance 



This picture shows a cube with one edge marked. How Percentage international 

many edges does the cube have altogether? correct ranking 

Lower grade 39 9 

Upper grade 44 10 



edge 




A. 6 

B. 8 

C. 12 

D. 24 



Relations and functions 



Example 1 1 : True statement of ages 

Henry is older than Bill, and Bill is older than Peter. 
Which statement must be true? 

A. Henry is older than Peter. 

B. Henry is younger than Peter. 

C. Henry is the same age as Peter. 

D. We cannot tell who is oldest from the 
information. 



Scotland's performance 

Percentage International 
correct ranking 

Lower grade 58 11 

Upper grade 65 14 
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Example 1 2: Operation to get B from A 



Scotland's performance 



What do you have to do to each number in Column A to 
get the number next to it in Column B? 



Column A 


Column B 


10 


2 


15 


3 


25 


5 


50 


10 



Percentage International 
correct ranking 

Lower grade 24 13 

Upper grade 31 18 



A. Add 8 to the number in Column A. 

B. Subtract 8 from the number in Column A. 

C. Multiply the number in Column A by 5. 

D. Divide the number in Column A by 5. 
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2.3 Scottish features and international comparisons 
in mathematics 



Characteristics 
of the schools 



The average number of class teachers 
per school was 13. 

Over 63% of the teachers had been at 
the school for more than 5 years. 

Just over 300 pupils on average 
attended the schools. 

On average just under 12 computers 
were available for use by teachers or 
pupils. 

Average class size was just under 26. 

Under 5% of the pupils were absent on 
a typical day. 



This was just below average. 

This was slightly higher than average. 
This was considerably below average. 
This was considerably above average. 

This was slightly higher than average. 

This was considerably higher than 
average. 



Characteristics of Over 90% of the teachers were female. This was well above average. 

teachers 

Teachers had been teaching for between This was less than in most other 
13 and 15 years. countries. 



Mathematics teaching 
practices 



Teachers spent between 240 and 260 
minutes per week on mathematics. 

Between 79 and 85% of teachers 
always divided their classes into groups 
for mathematics. 

Between 68 and 73% of teachers 
allowed almost all pupils access to 
calculators in mathematics lessons. 

1 1 % of teachers never set mathematics 
homework. 

Over 25% of pupils reported that they 
spend no time studying mathematics or 
doing mathematics homework. 

34 to 36% of teachers said that having 
pupils of different academic ability 
limited their teaching a great deal. 



This was above average. 

This was much higher than in any 
other country. 



This was higher than in any other 
country. 



This was higher than in all but 3 or 4 
other countries. 

This was more than in all but 1 or 2 
other countries. 



This was higher than in all but 2 or 3 
other countries. 
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Mathematics teaching 
practices (cont'd) 



Characteristics of pupils 



0 




14 to 16% of teachers said a high pupil- 
teacher ratio limited their teaching a 
great deal. 

93% of schools provided remedial 
teaching in mathematics. 

Just over 70% of schools provided 
enrichment activities in mathematics. 

91 to 95% of pupils reported using 
calculators in most or some 
mathematics lessons. 

67 to 73% of pupils reported using 
computers in most or some mathematics 
lessons. 

93% of pupils reported working in pairs 
or small groups in most or some 
mathematics lessons. 

Over 88% of pupils reported using 
things from real life to solve 
mathematics problems in most or some 
lessons. 



About 30% of pupils reported having 
over 200 books at home. 

83 to 89% of pupils reported having a 
computer at home. 

Between 23% and 27% of pupils 
reported watching TV or videos for 
more than 3 hours per day. 

1 3 to 1 6% of pupils reported playing 
computer games for more than 3 hours 
per day. 
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This was less than average. 



This was slightly higher than average. 

This was slightly above average. 

This was higher than any other 
country. 

This was higher than any other 
country. 

This was higher than in any other 
country. 

This was well above average. 



This was slightly above average. 

This was higher than in all but one 
other country. 

This was about average. 

This was higher than in all but one 
other country. 
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Pupils' views 84 to 86% of pupils said they liked 

mathematics or liked it a lot. 

Over 97% of pupils reported that their 
mothers thought it was important they 
did well in mathematics. 

97% of pupils reported that they 
thought it was important they did well 
in mathematics. 





This was just above average. 



This was well above average. 



This was slightly above average. 
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3 . 1 Performance in science 



Comparisons are made between the science 
performance of Scottish pupils and pupils in other 
countries which participated in TIMSS. Some of 
these comparisons are of scaled scores derived using 
plausible values procedures which are described in 
the second TIMSS Technical Report. Thecomparisons 
are made of: 

□ overall mean scores 

□ mean scores in the four reporting 
categories 

□ mean scores on tasks identified in the 
curriculum matching exercise 

□ performance on selected tasks 

Separate mean scores are shown for the two 
stages which contained the most 9 year old pupils. In 
Scotland these stages were primary 4 and primary 5. 
TIMSS also calculated median scores for pupils who 
were 9 years old at the time of testing. Where 
appropriate, comparisons are made with results from 
the International AssessmentofEducational Progress 
carried out in 1991. 

Overall mean scores 

The international mean scores for the lower and 
upper grades in all countries are presented in Table 4. 
The international mean for the lower grade was 473, 
compared to Scotland’s 484, and for the upper grade 
it was 524, compared to Scotland’s 536. Overall 
Scotland’ s ranking was 1 2th out of 24 countries at the 
lower grade and 1 3th out of 26 countries at the upper 
grade, including those countries which did not meet 
TIMSS sampling criteria. 

Two of the Pacific rim countries did well, Korea 
and Japan being ranked (on lower grade scores) first 
and second. Two others, Singapore and Hong Kong, 
did less well, being ranked 10th and 1 3th respectively 
at the lower grade. Thailand did considerably worse, 
being ranked 2 1 st at the lower grade. 

Of the European countries, Austria did well (and 



Table 4: Mean scaled scores in science at each 
grade 



Country 


Mean Score 


Mean Score 




Lower Grade 


Upper Grade 


Korea 


553 


597 


Japan 


522 


574 


United States 


511 


565 


Australia 


510 


562 


Austria 


505 


565 


Netherlands 


499 


557 


England 


499 


551 


Czech Republic 


494 


557 


Canada 


490 


549 


Singapore 


488 


547 


Scotland 


484 


536 


Hong Kong 


482 


533 


Ireland 


479 


539 


New Zealand 


473 


531 


Latvia 


465 


512 


Norway 


450 


530 


Greece 


446 


497 


Iceland 


435 


505 


Portugal 


423 


480 


Cyprus 


415 


475 


Iran 


356 


416 


Countries not meeting TIMSS sampling criteria 


Slovenia 


487 


546 


Hungary 


464 


532 


Israel 


- 


505 


Kuwait 


- 


401 


Thailand 


433 


473 



was ranked 5th at the lower grade and 3rd at the upper 
grade) and Scotland, England, The Netherlands and 
Ireland did better than average. Norway, Greece, 
Iceland, Portugal and Cyprus were significantly poorer 
than Scotland. 

Two of the eastern European countries were 
better than average, namely the Czech Republic and 
Slovenia (ranked 8th and 1 1th at the lower grade). 
Latvia was a little below average and Hungary was 
significantly poorer than Scotland. 
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Other English speaking countries varied in their 
performance. The USA and Australiadid well (ranked 
3rd and 4th at the lower grade) and Canada was better 
than average. New Zealand was not significantly 
poorer than Scotland and was on the international 
mean score at the lower grade. 

Table 5 shows Scotland’s position relative to 
other countries. Scotland was significantly better 
than eight countries at the lower grade and nine 
countries at the higher grade, and was significantly 
worse than five countries at the lower grade and 
seven countries at the upper grade. There were no 
consistent differences between performances in 
TIMSS and IAEP, but there were indications that 
Hungary had done better in IAEP and Canada had 
done worse. 

Differences between the lower and 
upper grades 

The same tests were taken by each grade and the 
difference in scores gives an indication of the gains 
made in this year. In Scotland, the difference was 52 
score points, and Scotland was ranked 17th out of 24 
countries. The differences averaged 5 1 score points 
and ranged from 80 in Norway to 40 in Thailand. 
These gains are bigger than those between the two 



grades containing the most 13 year olds, which may 
be because there is a significant improvement in 
reading ability between the younger grades which 
helps the pupils in the upper grade to cope better with 
written questions. 

If the average gain score of 51 is regarded as a 
year’s progress, then only Korea was over a year 
ahead of Scotland. 

There were no significant differences from 
Scotland’ s mean score difference i n any of the content 
areas. 

Benchmarks of performance 

If we regard the mean scores over all countries for the 
top 10%, the top 25% and the top 50% of pupils as 
benchmarks of performance, we can compare the 
percentage of pupils in each country reaching these 
benchmarks. At the lower grade, 8% of Scottish 
pupils achieved the 10% benchmark, 24% the 25% 
benchmark and 49% the 50% benchmark, and 
Scotland’s rankings were 9th, 9th and 11th 
respectively. At the upper grade, 9% of Scottish 
pupils achieved the 10% benchmark, 22% achieved 
the 25% benchmark and 50% achieved the 50% 
benchmark, and Scotland’s rankings were 9th, 1 1th 
and 13th respectively. 



Table 5: Scotland's position relative to other countries 





Lower Grade 


Upper Grade 


Countries scoring significantly 
higher than Scotland 


Korea, Japan, USA, 
Australia, Austria 


Korea, Japan, USA, 
Australia, Austria, 
Netherlands, Czech Republic 


No significant difference 
from Scotland 


England, Netherlands, 

Czech Republic, Canada, 
Singapore, Slovenia, Hong Kong, 
Ireland, New Zealand, Latvia 


England, Canada, Singapore, 
Slovenia, Ireland, Hong Kong, 
Hungary, New Zealand, Norway 


Countries scoring significantly 
lower than Scotland 


Hungary, Norway, Greece, 
Iceland, Portugal, Cyprus, 
Iran, Thailand 


Latvia, Israel, Iceland, 
Greece, Portugal, Cyprus, 
Iran, Kuwait, Thailand 
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Curriculum matching exercise 

In the curriculum matching exercise, the TIMSS task 
bank was scanned by each country to identify the 
tasks which were within their curriculum. The 
performance of pupils was then assessed on the basis 
of the tasks judged to be within the curriculum of each 
country. The tasks rated as being within Scotland’s 
curriculum had score points of 33 for the lower grade 
and 45 for the upper grade, compared to 105 score 
points for all of the tasks. This indicates that relatively 
few tasks were rated as being within Scotland’s 
curriculum as defined by the 5-14 Environmental 
Studies guidelines. 

Scotland’s scores on the tasks within its 
curriculum were 61% at the lower grade, where its 
ranking was 12th (out of 22), and 69% at the upper 
grade, where its ranking was 10th (out of 24). This 
indicates that pupi Is at the lower grade did marginally 
worse on tasks rated as being within the Scottish 
curriculum than they did on all tasks and pupils at the 
upper grade did slightly better on the tasks within the 
Scottish curriculum. 

Performance of 9 year olds 

In 22 of the TIMSS countries, the percentage of 9 year 
olds in the two grades tested exceeded 75%. For these 
countries the median scores were considered to be the 
best indicator of the performance of the 9 year old 



pupils. Scotland’s ranking in these countries was 5th 
out of the 22 countries, which is a considerable 
improvement on the overall grade rankings due to the 
younger average age of the Scottish pupils. 

Performance in different aspects of 
science 

* 

The tests contained 97 science test items, of which 1 7 
were in earth science (more akin to geography in 
Scotland), 4 1 were i n life science, 30 were in physical 
science and 9 were in environmental issues and the 
nature of science. Scotland’s rankings were similar in 
all the aspects of science tested, although the lower 
grade seemed to do slightly better in environmental 
issues and the nature of science and the upper grade 
seemed to do worse on life science. These rankings 
out of 24countries at the lower grade and 26 countries 
at the upper grade are shown in Table 6. 

Gender and performance in science 

At the lower grade, boys performed better than girls 
in 20 countries (out of 24), but the differences were 
only significant in 9 countries. At the upper grade, 
boys performed better than girls in 22 countries (out 
of 25), but the difference was only significant in 10 
countries. In Scotland, boys did better than girls at 
both grades, but the differences were not significant. 



Table 6: Scotland’s ranking on aspects of science 





Earth 


Life 


Physical 


Environmental issues 




science 


science 


science 


and nature of science 


Lower grade ranking 


12 


13 


13 


10 


Upper grade ranking 


15 


17 


14 


13 




23 
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3.2 Examples of science test items 

Earth science 



Example 1 : River on plain: Good place for farming 

The diagram shows a river flowing through a wide plain. The plain 
is covered with several layers of soil and sediment. 



River Channel 




Scotland’s performance 

Percentage International 
correct ranking 

Lower grade 60 6 

Upper grade 72 7 



Write down one reason why this plain is a good place for farming. 



Example 2: Size of sun and moon 



Scotland's performance 



The Sun is bigger than the Moon, but they appear to be 
about the same size when you look at them from the Earth. 
Why is this? 



Percentage International 
correct ranking 

Lower grade 21 13 

Upper grade 36 9 



Life science 



Example 3: Function of the heart 

Write down one thing your heart does that helps the other 
parts of your body. 



Scotland’s performance 

Percentage International 
correct ranking 

Lower grade 37 6 

Upper grade 53 6 
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Example 4: Why eat fruits and vegetables 



Scotland's performance 



What is the BEST reason for including fruits and leafy 




Percentage 


International 


vegetables in a healthy diet? 




correct 


ranking 


A They have a high water content. 

B They are the best source of protein. 

C. They are rich in minerals and vitamins. 

D. They are the best source of carbohydrate. 


Lower grade 


61 


10 


Upper grade 


59 


16 



Environmental issues and the nature of science 

Example 5: Crowing seeds in light or dark 

To find out whether seeds grow better in the light or dark, 
you could put some seeds on pieces of damp paper and 

Lower grade 32 7 

A. keep them in a warm, dark place Upper grade 36 1 1 

B. keep one group in a light place and another in a 
dark place 

C. keep them in a warm, light place 

D. put them in a light or dark place that is cool 



Scotland's performance 

Percentage International 
correct ranking 



Example 6: Oil spills 



Scotland's performance 



Write down as completely as possible why large oil spills Percentage International 

in rivers and seas are harmful to the environment. correct ranking 

Lower grade 14 12 

Upper grade 23 15 
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Physical science 



Example 7: Class over a candle 



Scotland's performance 



When a glass jar is placed over a lighted candle, the flame goes out. 




Percentage International 
correct ranking 

Lower grade 45 1 5 

Upper grade 66 1 1 



Why does this happen? 



Example 8: Block floating in water 



Scotland's performance 



The picture shows a block of wood floating in fresh water. 



\ , / 



\ y 



Block of wood 
Fresh water 



Percentage International 
correct ranking 

Lower grade 29 1 2 

Upper grade 31 17 



If this block were placed in salt water from the ocean, 
which picture shows what would happen? 



A. 



\ / 



Salt Water 



vl y 



B. 



□ 






Salt Water 



C. 



\ , / 



V. 



Salt Water 



D - A 



Salt Water 
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3.3 Scottish features and international comparisons in 

science 



Science teaching practices 



Science was taught for between 59 and 
71 minutes per week at the lower 
grade and between 67 and 78 minutes 
per week at the upper grade. 

About 40% of teachers said their 
teaching was limited quite a lot or a 
great deal by high pupil teacher ratios. 

About 60% of teachers said their 
teaching was limited quite a lot or a 
great deal by having pupils with 
different academic abilities. 

Only 16 to 17% of teachers taught 
science as a separate subject. 

About 23% of schools provided 
remedial teaching in science. 

Between 20 and 24% of schools 
provided enrichment activities in 
science. 

About 90% of pupils reported that in 
most or some science lessons teachers 
showed how to do science problems. 

About 85% of pupils reported that in 
most or some science lessons they 
copied notes from the board. 

About 86% of pupils reported that in 
most or some science lessons they 
worked on science projects. 

About 84% of pupils reported that in 
most or some science lessons they 
worked alone from worksheets and 
textbooks. 

About 83% of pupils reported that in 
most or some science lessons they used 
things from life to solve science 
problems. 



This was less than in most other 
countries. 

This was lower than average. 

This was about average. 

This was lower than in all but two 
other countries. 

This was about average. 

This was slightly below average. 

This was just above average. 

This was about average. 

This was slightly above average. 

This was about average. 

This was above average. 
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Science teaching practices 
(cont'd) 



Pupils' views 



O 




About 82% of pupils reported that in 
most or some science lessons the 
teacher demonstrated experiments. 

About 73% of pupils reported that in 
most or some science lessons they 
carried out experiments. 

About 70% of pupils reported that 
teachers set homework in most or some 
lessons. 

About 60% of pupils reported that they 
did not study or do science homework 
on a normal school day. 



Over 90% of pupils agreed or agreed 
strongly that their mothers thought it 
important they did well in science. 

Over 90% of pupils thought it was 
important they did well in science. 

Just over 80% of pupils agreed or 
agreed strongly that they usually did 
well in science. 

Over 80% of pupils said they liked 
science or liked it a lot. 



This was just above average. 



This was above average. 



This was below average. 



This was more than in all but 3 other 
countries. 



This was about average. 



This was about average. 
This was below average. 



This was about average. 
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4 Conclusions 



□ Scotland’s performance in mathematics relative 
to the other TIMSS countries was poor. Our 
scores were below the international averages 
and we were placed in the lower half of 
countries in terms of our performance. There 
appeared to have been some deterioration in our 
performance since the IAEP study in 1991. 

□ Scotland’s performance in science was quite 
good. Our scores were above the international 
averages and we were placed among the top 
half of countries in terms of our performance. 
There was no indication of any deterioration in 
performance since the IAEP study in 1991. 

□ The top performing countries in mathematics 
were from the Pacific rim - Korea, Singapore, 
Japan and Hong Kong, but Thailand performed 
poorly. In science, Korea and Japan were again 
top performers, but Singapore and Hong Kong 
performed very similarly to Scotland. The USA 
and Australia did notably well in science. 

□ The gain in performance between primary 4 and 
primary 5 was just above the average gain for 
all countries in both mathematics and science. 

□ Because Scottish pupils were amongst the 
youngest in the study, the median performance 
of 9 year olds compared to other countries was 
substantially better than the performances of the 
two grade samples. 



□ Scottish pupils’ performance was better in some 
aspects of mathematics and science than in 
others, but performance in geometry was 
notably good. 

□ In most countries, boys scored higher than girls 
in both mathematics and science, but the 
differences were significant in far fewer 
countries. In Scotland, boys scored higher than 
girls in all but the upper grade in mathematics, 
but the differences were not significant. 

□ The time spent teaching mathematics in 
Scotland was above average, but the time spent 
teaching science was much lower than average. 

□ Pupils’ use of calculators and computers in 
mathematics lessons was higher than in any 
other country. 

□ Less homework is set in Scotland in both 
mathematics and science than in most other 
countries. 

□ Levels of TV and video watching in Scotland 
were about average, but pupils played computer 
games more than in almost all other countries. 
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